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Abstract 

In this paper, a new face recognition technique based on 
Hidden Markov Model (HMM), Pre-processing, and 
feature extraction (K-means and the Sobel operator) is 
proposed. Two main contributions are presented; the first 
contribution in the pre-processing were image’s edges are 
normalized to enhance the HMM models to be non- 
sensitive to different edges. The second contribution is a 
new technique to extract the image's features by splitting 
the image into non-uniform height depending on the 
distribution of the foreground pixels. The foreground 
pixels are extracting by using the vertical sliding 
windows. The proposed technique is faster with a higher 
accuracy with respect to other techniques which are 
investigated for comparison. Moreover, it shows the 
capability of recognizing the normal face (center part) as 
well as face boundary. 

Keywords: HMM, Sobel operator, Face Recognition 
, Accuracy. 

Nomenclature: 


DCT 

Discrete Cosine Transform 

HMM 

Hidden Markov Model 

LDA 

Linear discriminate Analysis 

ORL 

Olivetti Research Laboratory 

PCA 

Principal Component Analysis 

PDBNN 

Probabilistic Decision- Based Neural Network 

SVD 

Singular Values Decomposition 

Wo 

represents the width of overlapping of the 
vertical frames 


1. Introduction 

Face recognition is identifying a human face among other 
different faces. It is widely used in all recent applications 
particularly related to the security field. For example, 
patterns recognition techniques are very essential in the 
treatment of the facial images to identify the criminal act, 
ID and credit card verification. Face recognition has 
become necessary in commercial and law enforcement 
researches. Recently, dedicated work targets face 
recognition process, which is aiming to handle the trade- 
off between time and accuracy. However, face 
recognition faces too many challenges such as the 
difference between human faces in color, shape, and 


dimensions. Therefore, there is a need for image analysis 
to extract its landmarks and forms, and then convert it 
into data to be processed. There are two main categories 
used for face detection; the first category is the feature 
based where analysis is performed on the main geometry 
of the face like eyes, nose, ears, and mouth. The second 
category is a holistic base where the face is analyzed as a 
two-dimensional pattemfl]. A wide range of models can 
be used for addressing this issue, most commonly models 
are Geometrical Feature Matching [2], Eigenfaces [3], 
Neural Networks [4], Linear discriminate analysis (LDA) 
[5], Principal component analysis (PC A), Discrete cosine 
transform (DCT) [6] and the fuzzy logic technique [7], 
which is used for face detection and Hidden Markov 
Models. Geometrical feature matching model is based on 
extracting geometrical features to form a complete 
picture of the face. This model showed a low accuracy 
with 75% on ORL database recognition rate. Another 
model is Eigenfaces method which uses the 
Principal Component Analysis (PC A) with 90.5% 
recognition rate on ORL database. Although this model 
is simple and fast however it is not robust enough over 
the changes in face orientation and lighting conditions. A 
neural network is a widely used model in many pattern 
recognition applications. The paper [8] used Probabilistic 
Decision- Based Neural Network (PDBNN) for face 
recognition with 96 % recognition rate. The 
recognition problems of this model appear when 
increasing faces classes. Here a used stochastic process is 
the hidden Markov model. Hidden Markov Model 
(HMM) is used for developing features extraction 
methods and enhancing not only the operation of face 
detection but also the data processing time as will be 
shown in this paper. In this paper, we introduce a new 
method for pre-processing the input image and a new 
method of extracting features by splitting the input image 
horizontally with non-uniform heights. This paper is 
organized as follows. In section 2, we explain our 
proposed technique then, we present the pre-processing 
step for the input image in section A. In section B we 
explain the process of the feature extraction. In section 
C, we introduce the use of HMM in the recognition 
system. In section 3 we give a full description of the 
obtained simulation results followed by the conclusions 
and future work is highlighted in section 4. 
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2. The Proposed Technique 

Our proposed technique consists of multiple steps: 

• The pre-processing step which will be 
explained in the following section. 

• The feature extraction to prepare data to be 
processed. 

• Appling HMM on the recognition system. 

We calculate the accuracy (recognition rate) to measure 
the ability of recognize the image by the system.Figurel 
shows the block diagram for our proposed technique. 



Figure 1 Face Recognition Proposed Technique 


Pseudo code 

READ new input image 
notknown = 1 
WHILE (not known) 

DO median filter to reduce noise 
DO other enhancement 

DO sobel filter to detect horizontal and vertical edges 

IF (imagesize! = threshold size) 

DO resize the image to new size 
ENDIF 


DO segmentation of the resized image with overlapping 
sliding window (horizontal and vertical) 

DO extract features from the segments of the image 
DO KMEANS and Clustering 
IF (HMM training) 

IF (new class) 

DO add new class model 
ELSE 


DO modify HMM class model using the image 
ENDIF 


ELSE 

Classify the image using the HMM models 
IF recognized 

WRITE output the image with decision 
Break from while loop 

ENDIF 

ENDIF 

ENDWHILE 

A. Pre-processing 

The first step is to change the image from RGB format to 
its analogous grayscale format. A grayscale image is a 
monochrome image, which contains information about 
the brightness only without containing any information 
about the color. A grayscale data represents a wide range 
of intensities. The typical image contains 8 bit/pixel, this 
allows the image to be represented with different level of 
brightness (0-255) [9]. The pre-processing is an essential 
step to enhance the ease of features extraction and to 
reduce the noise in the image by using the median filter 
which is used to remove impulsive noise and decrease 
edges blurring. Pre-processing stage is important step in 
enhancing the performance of the whole system due to 
the following reason: 

• normalizing the image’s edges to enhance the 
HMM models and makes it not sensitive to different 
edges. The recognition rate of our system without the 
pre-processing stage is 97%, after pre-processing the 
recognition rate 100% (enhanced 3 %). 

B. Feature Extraction 

After the pre-processing stage, the input image in binary 
format in the ORL database is used to extract a feature 
vector, which labeled the background pixels by logic "0" 
and the foreground pixels by logic "1". The input image 
is segmented into n horizontal segments have the same 
number of foreground pixels in each one of them 
(segment). The total number of pixels in the image is 
counted and then divided by n which results in the 
required number of pixels per segment. Then the input 
image is divided into vertical overlapping frames. Each 
one of these frames is divided into fixed width small 
cells, while the cell's height depends on the pixel's image 
and the distribution of foreground pixels in the images. 
The sliding window is shifted along the image from right 
to left and a feature vector is calculated for each frame, 
as shown in Figure2. 


height (h ) 


Figure. 2 Feature extraction using a sliding window with 
overlapping. 
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The image’s height represents ( h ), the image’s width 
represents ( w ), ( Wo ) represents the width of 
overlapping of the vertical frames, and (Wv) represents 
the width of the vertical frame. The best scenario in our 
technique when ‘ Wv ’= 7 pixels with ‘ Wo ’= 4 pixels 
(trial and error), this sliding accomplished the 100% 
accuracy (recognition rate) as it extracted the maximum 
necessary features for recognition. 6 cells (trial and 
error) are used per vertical frame with non uniform 
heights to extract this feature for each cell, the number of 
" 1 " pixels are calculated to produce 6 features per frame 
(6 cells per frame xl feature per cell) as shown in 
figure3. 



Figure 3 Six features per frame by dividing the image into six 
non uniform heights. 


gradient direction. To extract the gradient features we 
used two cells with non-uniform heights per frame (trial 
and error). For each cell, eight gradient features are 
produced to form 16 features per frame (2 non-uniform 
cells per frame x 8 features per cell) as shown in figure 4 



[0 50 85 90 70 66 85 0 ] 
[0 60 43 66 95 88 20 0 ] 


Figure 4 Sixteen feature per frame 

Pseudo-codes for Sobel edge detection method 

Input: A Sample Image 

Output: Detected Edges 

Stepl : Accept the input image 

Step2: Apply mask Mx ,M y to the input image 

Step3: Apply Sobel edge detection algorithm and the 

gradient 

Step 4: Masks manipulation of Mx,M y separately on the 
input image 

Step 5: Results combined to find the absolute magnitude 
of the gradient 

Step 6: the absolute magnitude is the output edges 


• Gradient Features 


The gradient is a vector which has certain magnitude and 
direction. 


V/ = 


5f_ 

Sx 

S£ 

Sy 


Advantage of the sobel operator: 

I. Faster to compute gradient operator because 
Gaussian smoothing effect. 

II. Less sensitive to noise present in image and 
smoothing affects the accuracy of edge detection. 

The magnitude of gradiant provide information about the 
strength of the edges[10]. 

V/ = \| ( fx )2 + ( % }1 = ^ (Mx)2 + (My>1 (1) 

The Direction is per-pendicular to the direction of the 
edge V/ = tan _1 (My /Mt) (2) 


This is done by applying the Sobel operator on the image 
to extract the vertical and horizontal gradient components 
[6]. To compute the feature vector, only the direction is 
used, which can range from 0 to 360 degrees. This range 
is divided into 8 non-overlapping regions each of them is 
45 degree. In each sampling region at each pixel, a 
histogram of gradient directions is taken. In each region 
step, histogram value corresponds to the count of each 


• Clustering and K-mean 

Clustering technique is a well known tech, in the field of 
pattern recognition, image and data analysis. The concept 
of clustering can be defined as a process of creating a 
group of sample, each group contains number of sample 
which are similar, so this groups are called clusters. 
Many techniques of clustering are proposed using the 
hierarchical for clusters of a unit element [11]. 
Considering the two main factors (simplicity and 
efficiency), clustering technique are used for the natural 
image segmentation. In partitional clustering, our 
objective is to create one set of cluster, where data is 
divided among similar groups. In our work, K-means 
clustering approach is used to perform image 
segmentation using Matlab software. We use clustering 
to classify the set pixels that have same characteristics to 
be in one cluster. K-means clustering is easy to be 
applied in large datasets [12]. First we made to define 
one K-centroid for each cluster, which is considered the 
main idea. Then, when the centroid are organized in a 
cunning way as different locations well cause different 
results, so it is better to place centroid far away with 
respect to each other. After that each point related to the 
given data set is associated to the nearest centroid [13]. 
For each step a new K-centroids are required to be 
recalculated. Then a new binding has to be done between 
the nearest new centroid and the same data set points. A 
loop has been generated. In this paper, our results show 
that the k-centroid change their location in a different 


3 
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way until the k-centroid have a defined fixed location as 
shown in table 1. 

Pseudo-codes for K-means and clustering 

X\ a set of N data vectors Data set 

C : initialized k cluster centroids Number of cluster 

/ 

C: the cluster centroids of ^-clustering random initial 

centroids 

P = {p{i ) | i = 1, . . ., TV} is the cluster label of X 
KMEANS(X, C ) -► (C, P) 

REPEAT 
C <— C ; 

previous I 

FOR all / E [7, A] DO 

Generate new optimal paritions 
p(i) <— arg min d(x m9 c y); 

l <j <k 

FOR all j E [7, £] DO 

Generate optimal centroids 

c. <— Average of x., whose p(i ) =y; 

UNTIL C = C 

previous 

C. Hidden Markov Model 

HMM is a complex stochastic process, which needs to be 
done two times to assure the generation of an efficient 
model for the sequential data.HMM is a defined set of 
states, and transition probabilities control the transitions 
among those states. HMM process deals with all feature 
observations in all states and then produce probability 
distribution to indicate the confidence of the object 
represented by the model. The classification can be 
obtained from an unknown pattern Based on the extracted 
feature sequence by finding a model that generates a 
maximum probability [4]. The HMM used in this paper is 
a discrete HMM, the used HMM classifier is based on the 
same classifier implemented in HTK Recognition Toolkit 
[14]. It uses the Viterbi algorithm, which searches for the 
most likely sequence of a hidden state given the input 
feature vector. Three states HMM are depicted in flgure5, 
which shows that we allowed a transition to the current 
and the next states only. Two of these are emitting states 
and have their output probability distributions. The 
transition matrix for this model consists of 3 rows and 3 
columns, which represent the transitions between these 
states such that incase of transition, we will set the matrix 
element to one and matrix element will be set to zero in 
case of no transition. The element of HMM are: 

• TV is the number of state in the model. If S is the set 

of state, then s = {51,52, 53, *SV}.The state of the 

model at time t is given by (J_t e S, 1 < t < T ? where 
T is the length of the observation sequence 
(number of frames). 

• M, the number of different observation 

symbols. If V is the set of all possible 
observation symbols V= { Vi, V 2 , , Vm} . 


• A the state of transition probability matrix 
i.e. A= | a,ij} where 

ay = P[qt = Sj \ qt - 1 = Si] o) 

1 < i, j < N with the constraint , 0 < CU, j < 1 

and, 

N 



• B, the observation of symbol probability matrix i.e. 
B— \bj(k)} , where 

bj(k) = P[Ot = vk\q t = S] (4) 

l<j<N,l<K<M and Ot is the observation 
symbol at time t . 

• 71, the initial state distribution, i.e. = j 7T?j where 

7ii=P [qi=Si] , 1 < i < N ( 5 ) 

Using a notation, a HMM is defined as the triplet 

X=(A,B ,7i) (6) 

the above characterization corresponds to a discrete 
HMM. In a continuous density HMM, the state are 
characterization by continuous observation density 
functions. The most general representation of the model 
probability density function (PDF) is a finite mixture of 
the form: 

M 

bi(o)=^ CikN (O, jUik , U ik) (7) 

k = 1 

where, Cik is the mixture coefficient for the Kth mixture 
in state i. without losses of generality N(0, fiik , U ik) is 
assumed to be a Gaussian PDF with mean vector jdik and 
covariance U ik 


a 2 , 2 as, 3 



Figure 5 Three states (transition) left to right 
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3. Simulation Results 

To evaluate the performance of the proposed recognition 
technique, recognition trials have been conducted on 
available subsets of the benchmark ORL face database. 
In our technique validation, we used 80 images for 
training purpose (8 persons x 10 images), and 40 images 
for testing purpose (8 persons x 5 images). The face 
recognition technique presented in this paper was 
developed, trained, and tested using Matlab 7. 10. The 
computer was a Windows 7 machine with a 2.1 GHz 
Intel core 2 processor and 2.87 GB of RAM. The 
following parameters have been determined through 
validation: Number of states, and the number of chosen 
centroids. The following table 1 shows the effect of 
changing centroids and stats versus time. The following 
fig. 6 shows samples of ORL database. Figure 7 and 8 
shows the effect of the given data in tablet . Table 2 and 3 
shows the comparison ORL database with different 
technique in Recognition rate and training time. 
Figure .9,10 and shows the effect of the given data in 
table 2and 3. Figure 11 shows the Comparison Results of 
Different Face recognition algorithms (Testing stage). 



Figure 6 Sample of ORL face database. 


Table 1 Shows the effect of varying the number of 
centroids and stats on the results. 


Model 

Recognition 
Rate (% ) 

Training 

Time/ image 

Proposed 

Technique 

100% 

0.00329 sec 

PDBNN 

96% 

20 min 

n-tuple 

86% 

0.9 

1DHMM + 
Wavelet 

100% 

1.13 sec 

Pseudo-2D 

HMM 

95% 

1 sec 

DCT-HMM 

99.5 % 

23.5 sec 

ID HMM + 
SVD 

99% 

0.63 sec 



Figure 7 Time (ms) versus the number of centroids 



Figure. 8 Accuracy versus the number of centroids. 


Table 2 Comparison ORL database with different technique in 
Re cognition rate and training time. 


#Centroids 

Average 

Time/Image(sec) 

Recognition rate 
(Accuracy %) 

6 

0.00326 

75 

8 

0.00329 

87.5 

10 

0.00346 

75 

12 

0.00427 

87.5 

14 

0.00329 

100 

16 

0.00336 

100 



Figure 9 Comparison Results of Different Face recognition 
algorithms and proposed technique. 
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The recognition rate of the proposed technique in this 
paper is higher than that of the traditional algorithms 
based on HMM. Also, we compare our proposed 
technique with different face recognition methods such as 
DCT-HMM, FisherFace, Laplacianfaces, ML-HMM, 
MCE-HMM,Pseudo-2D HMM,PDBNN and MC-HMM 
[15], our proposed method gives the highest recognition 
accuracy for all the sample sets. Comparison result is 
summarized in the following Table2 and figure. 9 and 
figure 10. 

Table 3 Comparison results of different face 
RECOGNITION ALGORITHM 


Models 

Recognition rate (%) 

Proposed Technique 

100 

E-HMM 

99.16 

Eigenfaces 

85.9 

Fisherfaces 

92.3 

Laplacianfaces 

93.2 

ML-HMM 

88.2 

MCE-HMM 

93.3 

MC-HMM 

97.5 



Figure 10 Comparison Results of Different Face recognition 
algorithms and proposed technique. 



Number of labeled samples 

Figure 1 1 Comparison of results of different face recognition 
algorithms (Testing stage). 


It is worth mentioning the technique has been 
implemented without performing the pre-processing step 
mentioned in section A. The results for four different sets 
of databases showed that the recognition rate is about 
97 %. This implies clearly the effect of the pre- 
processing step on the recognition rate. 


m 


4. Conclusion 

In this paper, we introduced a new effective technique 
for image face recognition. We proposed our technique 
based on intelligent features using efficient operator 
(Sobel). The First main contribution pre-processing 
stage has improved the performance of the overall 
system by about 3 % by normalizing the image’s edges 
to enhance the HMM models and makes it not sensitive 
to different edges. Although k-means, sobel and pre- 
processing are important steps, they simplify the features 
and enhance the recognition rate as shown in results. The 
second main contribution is a new technique of splitting 
the image into non uniform heights which has improved 
the performance of the system, when compared with 
using fixed heights. Our proposed technique showed the 
best performance compared with the other tested 
techniques in the sense of speed and recognition rate. . 
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